The Interference Paradigm for Network Job Scheduling

In this paper a novel scheduling model based on resource contention called the interference paradigm is presented. Using this paradigm, a range of scheduling policy options can be expressed. A simulation study of several interference-based scheduling policies in a NOW environment suggests that this approach can produce reduced completion time and high job throughput for a sequential and parallel job mix. The preliminary results indicate that this paradigm is especially promising as the degree of network heterogeneity increases and the system load is high − precisely where traditional scheduling schemes perform poorly. attempt to reduce job completion time [11]. Condor runs in a local-area NOW environment. It will use only idle machines and will migrate a job from a machine if the workstation user begins to use this machine. By exploiting only idle resources, the system throughput achieved by Condor will be limited. Utopia, now called LSF, will support the scheduling of sequential and parallel jobs and assumes that all machines are shareable, unlike Condor [12]. Utopia is a more scalable system that runs on hundreds of workstations. It exploits job information to make the best scheduling decision for a particular job in an attempt to reduce job completion time. DQS supports a batch scheduling capability for both sequential and parallel jobs and is designed to provide high system throughput [13]. The job scheduling problem has also been studied for single homogeneous parallel supercomputers [14][15]. This is essentially a special-case of the general problem in which sharing is normally limited to space-sharing. The problem is further simplified by the presence of a single scheduling agent such as the NQS job queueing system. In the network environment scheduling is distributed, and both timeand spacesharing must be considered. A new network-based job scheduling paradigm that considers the impact of job scheduling on currently running jobs in both time and space has been investigated. This class of scheduling policies are known as interference policies. Interference is an increase in job completion time due to contention for computation and communication resources. Using the interference metric, a spectrum of job scheduling strategies can be expressed including policies that are optimized for high throughput, reduced completion time, or both. A set of interference policies have been studied in simulation to determine their throughput and completion time characteristics. The preliminary results indicate that interference appears to be an important component of an effective job scheduling policy that achieves high job throughput and reduced completion time in a heterogeneous NOW environment. This paper is organized as follows. Section 2.0 describes the network and job model and details the necessary assumptions. Section 3.0 introduces the interference paradigm and a set of interference-based policies. Section 4.0 describes the simulation results obtained with these policies. Section 5.0 provides a summary and future work.